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Abstract — We consider Slepian-Wolf (SW) coding of 
multiple sources and extend the packing bound and the 
notion of perfect code from conventional channel coding to 
SW coding with more than two sources. We then introduce 
Hamming Codes for Multiple Sources (HCMSs) as a poten- 
tial solution of perfect SW coding for arbitrary number of 
terminals. Moreover, we study the case with three sources 
in detail. We present the necessary conditions of a perfect 
SW code and show that there exists infinite number of 
HCMSs. Moreover, we show that for a perfect SW code 
with sufficiently long code length, the compression rates 
of different sources can be trade-off flexibly. Finally, we 
relax the construction procedure of HCMS and call the 
resulting code generalized HCMS. We prove that every 
perfect SW code for Hamming sources is equivalent to a 
generalized HCMS. 



I. Introduction 

SW coding refers to lossless distributed compression 
of correlated sources. Consider s correlated sources 
X±,X2,--- ,X S . Assuming that encoding can only be 
performed separately that s encoders can see only one of 
the s sources but the compressed sources are transmitted 
to a base station and decompressed jointly. To the 
surprise to many researchers of their time, Slepian and 
Wolf showed that it is possible to have no loss in sum 
rate under this constrained situation (Q. That is, at least 
in theory, it is possible to recover the source losslessly 
at the base station even though the sum rate is barely 
above the joint entropy H (X±, X2, • • • , X s ). 

Wyner is the first who realized that by taking com- 
puted syndromes as the compressed sources, channel 
codes can be used to implement SW coding [2]. The 
approach was rediscovered and popularized by Prad- 
han et al. more than two decades later (31 • Practical 
syndrome-based schemes for S-W coding using channel 
codes have then been studied ing^fD.fgMTl.QD.ED, 
iflOll . ifTTI HI, HI, HI, HSl- However, most work 

R. Ma was with the Department of Mathematics at the Hong Kong 
University of Science and Technology, Hong Kong. 

S. Cheng is with the School of Electrical and Computer Engi- 
neering, University of Oklahoma, Tulsa, OK, 74135 USA email: 
samuel.cheng@ou.edu. 

A part of this work was presented in DCC 2010 and was submitted 
to ISIT 2010. 

Manuscript received January 19, 2010 



is restricted to the discussion of two sources (3J, fRfl , 
H3, HI, HI except few exceptions BUI. Bl. EH. In 
this paper, we describe a generalized syndrome based 
SW code and extend the notions of a packing bound 
and a perfect code from regular channel coding to SW 
coding with arbitrary number of sources. Moreover, we 
introduce Hamming Code for Multiple Sources (HCMSs) 
as a perfect code solution of SW coding for Hamming 
sources (c.f. Definition [4]) and show that there exists 
infinite number of HCMSs for three sources. We then ex- 
tend HCMS to a more inclusive form dubbed generalized 
HCMS and show the universality of generalized HCMS. 
Namely, any perfect SW code of Hamming sources can 
be reduced to a HCMS code through equivalent operation 
(c.f. Theorem |4]). 

The paper is organized as follows. In the next sec- 
tion, we will describe the problem setup and introduce 
definitions used in the rest of the paper. In Section [Till 
we will present a major lemma useful to the rest of 
paper. We will introduce HCMS in Section|IVl In Section 
Ivl HCMS for three sources will be discussed in detail. 
Necessary conditions for perfect code will be given and 
the existence of HCMS for three sources will be shown. 
In Section [VH we extend HCMS to generalized HCMS 
using the notion of row basis matrix as defined in Section 
Ull In Section IVIIl we will show the universality of 
generalized HCMS. 

II. Problem Setup 

We will start with a general definition of syndrome 
based SW codes with multiple sources ET1 . 

Definition 1 (Syndrome based SW code). A rate 
( r ii r 2r - - > r s) syndrome based SW code for s cor- 
related length-n sources contains s coding matrices 
H\,H2, ■ ■ ■ , H s of sizes mi x n, ttt-2 x n, ■ ■ ■ , m s x n, 
where r« = rrii/n for i = 1, 2, • • • , s. 

• Encoding: The i th encoder compresses length-n in- 
put Xj into yi = Hi^i and transmit the compressed 
mi bits (with compression rate rj = rrii/n) to the 
base station 

• Decoding: Upon receiving all y^, the base station 
decodes all sources by outputting a most proba- 
ble xi , X2 , • • • , x s that satisfies H{Xi = yi,i = 
1,2,--- ,s. 
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For the ease of exposition, we will occasionally 
refer a compression scheme with its coding matrices 
(H±,--- ,H S ) directly. Moreover, let us introduce the 
following definitions. 

Definition 2 ((s, n, M)-compression). We refer to 
(s, n, M)-compression as a SW code of s length-ra 
source tuples with M total compressed bits (M = 
mi + m 2 H h m s ). 

Definition 3 (Compressible), we will say the set of s- 
terminal source tuples S to be compressible by a SW 
code if any source tuple in S can be reconstructed loss- 
lessly. Alternatively, we say the SW code can compress 
S. 

Apparently, a SW code can compress S if and only if 
its encoding map restricted to S is injective (or 1-1). 

At one time instance, we call the correlation among 
different sources a type-0 correlation when all source 
bits from different terminal are the same. In general, we 
call the correlation a type-t correlation if all source bits 
except t of them are the same. For highly correlated 
source, we expect that the most probable sources are 
those with type-0 correlations for all n time instances, 
and the next most probable sources are those with n — 1 
type-0 correlations and one type-1 correlation. We call 
these sources s-terminal Hamming sources of length n. 
Let us summarize the above in the following. 

Definition 4 (Hamming sources). A s-terminal Ham- 
ming source of length n is s length-n source tuple 
that contains either 1) entirely type-0 correlations for n 
time instances; or 2) type-0 correlations for n — 1 time 
instances and one type-1 correlation. 

Let S be the set containing all s-terminal Hamming 
sources of length n. By simple counting, the set S has 
size (s'n + l)2 n , where s' = s when the number of 
terminals s > 2 and s' = 1 when s = 2. Thus, if 
S is compressible by a SW code with C denoted as 
the set of all compressed outputs, we have a packing 
bound given by \C\ > (s'n + 1)2™. We call the code as 
perfect if the equality in the packing bound is satisfied 
(i.e., \C\ = (s'n + l)2 ra ). The notion of perfectness can 
be generalized to any set S of interests: 

Definition 5 (Perfect SW codes). A SW code is perfect 
if and only if |C| = \S\. 

In other words, the encoding map restricted to S is 
surjective (and injective also since S is compressible) if 
the compression is perfect. For the rest of the paper, 
let us restrict S to denote the set containing all s- 
terminal Hamming sources of length n. Note that if 



H\, -- ,H S can compress S, the intersection of null 
spaces of Hi, ■ ■ ■ ,H S should only contain the all-zero 
vector. Otherwise, let x belong to the intersection and 

s 

thus the s-tuple (x, • • ■ , x) £ S will have the same 

s 

syndrome (all-zero syndrome) as (0, • • • ,0) € S. This 
contradicts with the assumption that Hi, - ■ ■ ,H S com- 
presses S. 

The following definitions are used to simplify the 
subsequent discussion. 

Definition 6 (Hamming Matrix). An m-bit Hamming 
matrix (of size m x (2 m — 1)) consists of all nonzero 
column vectors of length m. Note that the parity check 
matrix of a Hamming code is a Hamming matrix. 

Definition 7 (Surjective Matrix). A surjective matrix is 
a full rank fat or square matrix. 

Definition 8 (Row Basis Matrix). Given a matrix A, we 
say a surjective matrix B is a row basis matrix of A, 
if row(B) = row(A), where row(A) denotes the row 
space of A, i.e., all linear combinations of rows of A. 

Example 1 (Row Basis Matrix). ( J is a row basis 
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Remark 1 (Row Basis Matrix "Transform"). There is 
a unique matrix C s.t. A = CB (because every row of 
A can be decompose as a unique linear combination of 
B since B is full rank). And there exists matrix D s.t. 
B = DA but D is not necessary unique. Thus, given 
a vector v, if we know Av, we can compute Bv (as 
DAv). Similar, we have Bv given Av. 

III. Null Space Shifting 

We will now introduce an important lemma that 
provides a powerful tool for our subsequent discussion. 
In a nutshell, the lemma tells us that it is possible to 
tradeoff the compression rates of different source tuples 
by shifting a part of the null space of a coding matrix 
to another. 

Our proof is based on analysis of the null spaces 
of the coding matrices. More precisely, we isolate the 
common space shared by all except one null spaces and 
decompose the null spaces as the direct sum (©) of the 
common space and the residual space. 

Lemma 1. Suppose Hi, - - ,H S can compress S and 
null(Hi) = K © Ni for all i but a r that null(H r ) = N r , 
then matrices H[, ■ ■ ■ , H' s , with null(H'^) = K © Ni for 
all i but a d that null(Hd) = can also compress S. 
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Furthermore, if all H'- are onto and (Hi, ■ ■ ■ , H s ) is a 
perfect compression, i.e., (Hi, • • • , H s ) restricted to S is 
bijective, then (H[, ...,H' S ) is also a perfect compression. 

Before the proof, we first notice that N r H K = {0}. 
Otherwise H\ , • • • , H s have common nonzero null vec- 
tor that contradicts with S being compressible by the 
code. So the notation K © N r is justified. 

Proof: We have nothing to prove if r = d. For 
r ^ d, we can simply put r = 2 and d = 1 without 
losing generality. 

Define S+ as S + S = {si +s 2 |si, s 2 G S}. Note that 
1) (H[,--- ,H' S ) restricted to S is 1-1 O if 
(mi,-- - ,m s ) G S+ and (H[mi,--- ,H' s m s ) = 
(0,--- ,0), then (mi,- ,m s ) = (0, • • • ,0). 
So let (mi, • • • , m s ) G <S+ and (H[mi, ■ ■ ■ , H' s m s ) = 
(0, • • • , 0). By checking back the null spaces of H'-, we 
have mi = ni, m 2 = k 2 + n 2 , mj = kj + rij, where 
rij G Nj, and {ki, k 2 } C K. So, mi + k 2 = ni + k 2 , 
m 2 + k 2 = n 2 , and rrij + k 2 = k 2 + kj + rij. By 
checking the null spaces of Hi, we find (mi, • • • , m s ) + 
(k 2 ,-- - ,k 2 ) G (nulLffi,--- ,wl\[H s ). As (k 2 ,--- ,k 2 ) 
has all type-0 correlations, (mi, • • • , m s ) + (k 2 , • • • , k 2 ) 
is also in S+. Since (Hi,-- - ,H S ) restricted to S is 
injective, by 1), we have m,, = k 2 for all j. In particular, 
mi = k 2 = ni, which implies k 2 G Ni n K = {0}. 
That is, k 2 = and thus m, = for all j. By 1) again, 



(K 



,H' S ) restricted to S is injective. 



For the second part, (Hi 



Ha) restricted to S is 



now surjective as well and hence (Hi, ■ ■ ■ ,H S ) per se is 
also surjective. Therefore all Hj are full rank matrices. 
These imply all H'- have to be full rank as well. Fur- 
thermore, if all all Hj are full rank, the dimension of the 
target space of (H[, ■ ■ ■ ,H' S ) = ns-^ =1 |null(^)| = 
ns — Ylj=i l n ull(-Hj)|, which equals to the the di- 
mension of the target space of (HI,-- - ,Hs). Since 
(Hi, ■ ■ ■ ,H S ) restricted to S is bijective, the dimension 
of the target space of (Hi, • • • ,H S ) (and hence that of 
(H'i, ■ ■ ■ , H' s )) is \S\. Since (H[, ■ ■ ■ , H' s ) restricted to 
S has been proven to be injective already, it must be 
bijective as well. ■ 

IV. Hamming Code for Multiple Sources 

Recall that S is denoted as the set containing all s- 
terminal Hamming sources of length n. Let M = mi + 
m 2 + • • • + m s be the total number of compressed bits. 
Then we have \C\ = 2 M and thus when s > 2, the 
equation for perfect compression becomes 



2 n (sn + 1) 



(1) 



Since sn + 1 = 2^ M n \ s obviously cannot be even. 
On the other hand, by Fermat's Little Theorem, we have 



1 = 2( s_1 ) (mod s) for every odd prime s > 1. This 
gives an infinite number of solution to dH). 

Now, we will present the main theorem that leads to 
HCMS. 

Theorem 1 (Hamming Code for Multiple Sources). For 

positive integers s,n,M satisfy (0 and s > 2, let P be 
a Hamming matrix of size (M — n) x (2 M ~ n — 1) = 
(M — n) x (sn). 

If P can be partitioned into 



P = [Qi,Q2,-~ ,Qs] 

such that each Qi is an (M — n) x n matrix and 

Qi + Qi + -- - + Q S = 0, 



(2) 



(3) 



and 



R 



( Qi \ 

Qs-i 

V T J 



(4) 



is invertible for some arbitrary T, then we have a set of 
s parity check matrices 

'GA (G 2 \ (G s 
jQiJ'^J' '"'\Q, 

that forms a perfect compression, where G\, G 2 , 
be any kind of row partition of T. That is, 

fGi\ 

G 2 



(5) 

G s 



T 



(6) 



\GsJ 



and some G, can be chosen as a void matrix. 



Proof: 

For any b, V{ G s.t. \v\ \ + |« 2 | H h \v s \ < 1, the 

input of correlated sources [b + v%, b + i> 2 , • • • , b + v s ] 
will result in syndrome 

Gi(b + vi)\ (G 2 (b + v 2 )\ (G s (b + v s 
Qi(b + vi)J ' \Q 2 (b + v 2 )J ' ' \Q s (b + v s ) / 

to be received at the decoder. The decoder can then 
retrieve (vi, - ■ ■ ,v s ) since 

Qi(b + vi) + Q 2 (b + v 2 ) + --- + Q s (b + v s ) 
=Qi(vi) + --- + Qs(v s ) (by®) 



-p 



V2 



\Vs) 



(by ©) 



and P is bijective over the set of all length-sn vectors 
with weight 1. 
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After knowing (v i, • • • , v s ), we can compute 
Gi(b), ■ ■ ■ , G s (b) and Qi(b), ■ ■ ■ , Q s {b). Thus, we have 



( Qi\ 

G 2 
V G s J 



(b) 



( Qi \ 

Q2 

Qs-i 
V T ) 



(b) = R(b). 



As R is invertible, we can recover b and hence the 
correlated sources [b + V\, b + v 2 , ■ ■ ■ ,b + v s ]. 

m 

Remark 2 (SW coding of three sources of length-1). 
Apparently, HCMS only exists if the (s—l)(M—n) < n, 
otherwise the required height of T will be negative. For 
example, let s = 3, n = 1, M = 3. Even though the 
parameters satisfy £T|), we will not have HCMS because 
n— (s— \){M— n) = —3. However, a perfect (trivial) SW 
code actually exists in this case, the parity check matrices 
for all three terminals are simply the scalar matrix (l). 

From Remark |2j we see that HCMS cannot model 
all perfect codes that can compress s-terminal Hamming 
sources. It turns out that we can modify HCMS slightly 
and the extension will cover all perfect SW codes for 
Hamming sources. We will delay this discussion to 
Section IVIIj In the next section, we will first discuss 
HCMS for three sources in detail. 

V. HCMS for Three Sources 

A. Necessary Conditions 

Now, let us confine to the case with only three 
encoders, i.e., s = 3. Let (Hi, H 2 , H 3 ) be a perfect 
compression for S. We have 



nulLffi n nullS 2 n null# 3 = {0}. 



(7) 



Hence we can decompose the null spaces into U © 
K 2 © K 3 , V © Ki © K 3 , W © Ki © K 2 , where Ki = 
wOHj n nulls*, (i,j, k) G {(1, 2, 3), (2, 3, 1), (3, 2, 1)}. 
Notice that (U © K 2 ffi K 3 ) D K x = {0} by ©. We 
also have (U © K 2 © K 3 © K x ) n W = {0} (because 
Vu E [/ ,Vki G Ki , Vk 2 G K 2 , Vk 3 G K 3 , and 
Vw G W, u + ki + k 2 + k 3 = w u + k 2 + k 3 = 
ki + w G nulliii (~1 nullii" 3 = K 2 ki = and 
w = 0). By the symmetry among U, V, W, we also have 
(C ® A)r\B = {0} for any A,B £ {U, V, W} and 
A 7^ B, where C = K x © K 2 © K 3 . By Lemma 1, the 
perfect compression is equivalent to full rank matrices 
with null space C © U, C © V, and W. 



Let us denote the dimensions of U, V, W, and C, as 
u,v,w, and c, respectively. Then we have the following 
lemma. 

Lemma 2. With u,v,w, and c described above, if the 
code can compress S and is perfect, then 3n — 2M < 
c < 3n — 2M + 3, where M = mi + m 2 + m 3 . Moreover, 
M - n - 2 < u,v,w < M - n. 

Proof: Even if the rest of the sources are known 
exactly, the correlation specified by S implies that there 
can be n + 1 possibilities for the remaining source. 
Therefore, 2 mi > n + 1 for i = 1,2,3. Denote 
Di as the dimension of the null space of Hi. Then, 
Di = n — rrii < n — log 2 (n + 1) < n — log 2 (n + 1/3) = 
n — log 2 (3n+l) + log 2 3. From the packing bound, if the 
code is perfect, we have 2 M = (3n + l)2 n . Therefore, 
D <n — (M — n) + log 2 3 and thus D < 2n - M + 1, 
where the second inequality holds because D, n, and M 
are all integers. 

Given the three null spaces to be C®U, C@V, and W, 
then we have c+v < 2n— M+l, c+u < 2n— M+l, and 
c + w < 2n — M + 1. Note that the last inequality holds 
since Lemma Q] tells us that full rank matrices with null 
spaces C © U, V, C © W also compress S perfectly. 
Moreover, since (C © A) n B = {0} for any A,B<E 
{U, V, W} and A ^ B, we have c+u+v < n, c+u+w < 
n, and c + v + w < n. And the total dimensions of the 
null spaces = 3n — M = c + u + c + v + w. Thus 
3n — M — (c + u + v) = c + w and 2n — M < c + w. 
Similarly, we have 2n — M < c+u and 2n — M< c+v. 
In summary, we have 



2n - M < c + a < 2n - M + 1, 



(8) 



a G {u, v, w}. Since w = 3n — M — (c + u) — (c + v), by 
®, 3n-M-2(2n-M+l) < w < 3n-M-2(2n-M) 
and thus M — n — 2 < w < M — n. Similarly, we have 
M -n-2<u,v < M -n. 

Now, substituting M — n — 2 < w < M — n back to 
©, we have 3n - 2M < c < 3n - 2M + 3 as desired. 

■ 

To summarize from Lemma |2j given n and M, there 
are only four cases for different values of c, u, v, and w 
assuming u > v > w as shown in Table U 

Moreover, the perfectness condition, 2 M = (3n+l)2 n , 
turns out to be restrictive enough to confine M and n 
into some limited possibilities as to be describe in the 
following Lemma. 

Lemma 3. All positive integers n and M that satisfy 
2 M = 2 n (3n + 1) have the forms (2 2a - l)/3 and 2a + 
(2 2a — l)/3, respectively, for some positive integer a. 
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TABLE I 

Feasible values of c, u, v, and w 



c 


it 


f 


w 


3n — 


2M + 


3 


M-n- 


2 


M — n — 


2 


M -n- 


2 


3n — 


2M + 


2 


M-n- 


1 


M — n — 


1 


M — n — 


2 


3n — 


2M + 


1 


M - n 




M - n - 


1 


M -ri- 


1 


3n 


- 2M 




M - n 




M - n 




AI - n 





TABLE II 

Values of n,M, M -n,3n- 2M for a = 1, 2, 3, 4, 5. 



a 


n 


M 


M — 71 


3n - 2Af 


1 


1 


3 


2 


-3 


2 


5 


9 


4 


-3 




21 


27 


6 


9 


4 


85 


93 


8 


69 


5 


341 


351 


10 


321 



Proof: It is easy to verify that 2 2a = 1 (mod 3) and 
2 2a+1 = 2 (mod 3) for any positive integer a. Moreover, 
since both M and n are positive integers, 3?i+l = 2 M ~ n 
is a positive integer as well. This implies 2 M ~ n = 1 
(mod 3) and thus M — n has to be even. Let M — n = 2a 
for some positive integer a, then we can rewrite n and 
M as a: n(a) = (2 2a - l)/3 and M (a) = 2a + n(a). ■ 
It is interesting to point out that M has to be divisible 
by 3. It can be proved using simple induction and we 
will skip the proof here. 

Lemma 4. M(a) = (2 2a - l)/3 + 2a, a G Z+ <fe/wi«/ 
in Lemma \3\ is divisible by 3. 

We list in Table JI] the first five possible values of 
M and n. M — n and 3ra — 2M are also included for 
convenience. 

Note that for both n = 1 and n = 5, 3n - 2M = -3. 
Thus, only the first case described in Table U will be 
possible (i.e. c = 0). 

By Lemma [T] the null space contributed by C can be 
reallocated to different terminals arbitrarily and yet the 
resulting code will still compress S and be perfect. 

For example, for n = 21 and M = 27, possible values 
of c, u, v, w are 12, 4, 4, and 4, respectively. This results 
in an asymmetric code with mi = 21 — 12 — 4 = 5, 
m2 = 5, and 7713 = 17. If this code compresses S, we 
can reallocate 4 dimensions of null spaces each from Hi 
and H2 to H3 and result in a symmetric code that can 
compress S as well. 

From the above discussion, we see that c ~ 3n — 2M 
increases exponentially with a wherea u,v,w~M — n 
only increases linearly. Therefore, for sufficiently large 
n, c will always be large enough that we can rearrange 



any perfect code to another asymmetric perfect code of 
desired rates by allocating C among different coding 
matrices. 

B. Existence of HCMS 

We have not yet shown that any HCMS exists. Ac- 
tually, the authors are not aware of any prior work that 
reported perfect SW codes with more than two sources in 
the literature. From Remark[2l we see that HCMS cannot 
model the trivial case for a = 1 (n = 1 and M = 3) even 
though by definition the trivial code (coding matrices 
equal to scalar identity for all three sources) is perfect. 
For a = 2 (n = 5 and M = 9), HCMS also does not 
exist since n-(s-l)(M-n) = 5-2(4) = -3 < (c.f. 
Remark |2]). However, there is actually no perfect code 
exists at all in this case as concluded in the following 
proposition. 

Proposition 1 (No perfect code for a = 2). There does 
not exist perfect code for SW coding of three length-5 
sources. 

Proof: See Appendix. ■ 
Even though HCMS does not exist for a = 1 and 
a = 2, it is possible to show that HCMS exists for a > 3. 
In the following, we will first show that HCMS for three 
sources exists for a > 3 using Theorem Q] 

Proposition 2 (HCMS exists for a = 3). Now we will 
show that perfect compression exists for s = 3 and a = 
3. For a = 3, n = 21 and M = 27, we let 

/l 0000010000111011000 0\ 
010000110000100000111 
001000011000011101011 
000100001100010011110 ' 
000010000110101101111 

\0 00001000011001001101/ 

/0 0010110111101000111 l\ 
1 1 1 1 1 1 1 1 1 1 1 1 00 
010001111011100000101 
y 101000111101011100111 ' 
010100111110001011011 

\0 01010011111110101110/ 

and 

/l 0010100111010011111 l\ 
110010000111001111111 
011001100011111101110 
101100110001001111001 ' 
1 1 1 1 1 1 00 1 1 1 1 00 

\0 01011011100111100011/ 

It is a little bit laborious but straightforward to see 
that P = [Q1Q2Q3] is a 6-bit Hamming matrix and 
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Qi + Q2 + Q3 = 0. So we have (O and ([3]) already. For 
©, let U and V be the matrices obtained by truncating 
the last 9 columns of Q\ and Q 2 : 

/l 0000010000 1\ 
010000110000 
001000011000 
000100001100 
0000100001 10 
\0 00001000011/ 



u 



and 



V 



We have V 



/0 0010110111 1\ 
100010110111 
010001111011 
101000111101 
010100111110 
\0 01010011111/ 

[0| J 6 ] + KU, where 

fo 1 1\ 
10 10 
10 1 
10 10 
10 10 
\0 1 1 0/ 



K 



Hence | lL L I is a full rank matrix with pivots on the 
Mi) 

Q 2 is a 21 x 21 

OI/9]. By Theorem [7] 
form a perfect 



first 12 columns. Therefore, R 

invertible matrix where T = 
G\\ f G 2 \ 



coding matrices . 
compression with defined in ©. 



Theorem 2 (HCMS exists for all a > 3). For s = 3, 

there is a perfect compression for n and M that satisfy 

3n(a) + 1 = 2 2a , (9) 
M(a) - n(a) = 2a, (10) 

where a is any integer larger than or equal to 3. 

Proof: From Proposition [2j we have shown that 
there exists HCMS for three sources when a = 3. 
Now we will show by induction that there is perfect 
compression for all a > 3. Suppose we have a partition 
of a Hamming matrix P of size 2ax(2 2a -l) (formed by 
all non-zero length-2a column vectors) as P = [ABC] 

such that A + B + C = 0, and f° rms a full ran k 

matrix with pivots on the first 4a columns whenever 
3 < a < k. By Theorem \T\ perfect compression can 
be built by choosing T = [0, 4 (a) _ 4a ]. 



We will show that the statement is also true for k + 1. 
Let P be the 2/c-bit Hamming matrix and A, B, C be 
its partition with properties described above. Let u 
1\ M (\ 



() , , v = 1 1 1 , w = 1 1 ] . Let Ai, Bi, Ci be the i-th 



.4, 



B A 



Aj Aj Aj Aj 



u 



w 



column of A,B,C respectively We define 

'0 
u 

"0 


,w w u v 



B i B 3 B 3 B 3 







w u 



Cj Cj Cj Cj 



where j runs from 1 to n = (2 2k — l)/3. 

It is easy to verify that P + = [A + B + C + ] is a 2(A;+1)- 
bit Hamming matrix consisting of 3 + (4(2 2fc — 1)) = 
2 2 ( fc +!) _ 1 different non-zero column vectors of length 
2(k + 1). And obviously A + + B + + C + = as A + 
J B + C = 0andu + v + w = O. 

Lastly we permutated the columns of A + , B + and 
C + simutanteously (keeping their sum zero) such that 
the first 4(/c + 1) columns are 



.4, 



B A 



C 4 



Ax 


A 2 ■ 


• A 2k 




A 2 


A 3 


A4 





• 





V 


w 


w 


u 


Bi 


B 2 ■ 


• B 2k 


Bx 


B 2 




£4 











w 


u 


V 


w 


Ci 


c 2 ■■ 


• C 2 k 


Cx 


c 2 


c 3 


C4 





•• 





u 


V 


u 


V 



Notice that 



2/,- 



M 2 

dependent by induction assumption and 



A 



(11) 



are linear in- 



v w w u 
w u v w 



/0 1 1 1\ 
1110 
110 1 



column vectors of 



\i 1 1/ 

are also linear independent. Therefore the first 4(fc + 1) 

are linearly independent. Hence 

is a full rank matrix with pivots on the first 4(k + 
1) columns. By Theorem [T] perfect compression can be 
built by choosing T = [0, I n (fc+i)-4(fc+i)]- 

■ 

VI. Generalized HCMS 

Now, we will extend HCMS so that it will cover 
the trivial case described in Remark [2] The main idea 
of generalized HCMS is to "loosen" the condition in 
dU) using the notion of row basis matrices defined in 
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Section JI] Let Y, a d x n matrix, be a row basis 



matrix of 



/ Qi 

Q2 



where Qi,--- ,Q S is a partition 



\Qs-i, 

of a Hamming matrix satisfying (fSJ) and ©. Since Y 
is a surjective matrix, there apparently exists T s.t. 

R = is an n x n invertible matrix. 

Theorem 3 (Generalized HCMS). Let d, i = 1, • • • , s 

be any row partition of T as in © and Ci be a 
row basis matrix of Qi for i = 1, • • ■ , s. Then a set 



of parity matrices 



Gi 



G 2 



G s 



forms a 



compression for the set of s-terminal Hamming sources 
of length n. Moreover, the compression will be perfect if 
di + d 2 + ... + d s + (n — d) = M, where di is the number 
of row of Ci. 

Proof: Let | • | be the function that maps an element 
in Z 2 to its norm in Z by counting the number of 
nonzero components, e.g., 
V; e Z% s.t. |v x | + |v 2 | + 
correlated sources [b+vi,b+V2, 
in syndrome 



1,1,0,1)| = 3. For any b, 
• • + |v s | < 1, the input of 
, b+Vol will result 



Gi(b + vi) 
Ci(b + vi) 



G 2 (b + v 2 ; 

c 2 (b + V2 ; 



G s {b + v a 
C s (b + v s 



to be received at the decoder. Given Q(b + Vj) at the 
decoder, we can recover Qi(b + Vj) from Remark Q] 
The decoder can then retrieve (vi, • • • , v s ) since 



Qi(b + vi) 
=Ql(vi)+- 
/vi\ 

-p ^ 



+ Q 2 (b + v 2 ) + • • • + Q s (b + v s 
• + Qs(-v s ) 



C 3 = [1]. f^j = Y = [1] and T is void 

and hence Gi are void. So d\ = d 2 = ds = d = 1 
and d\ + d 2 + d% + (n — d) = M. So from generalized 
HCMS, we get perfect compression with coding matrices 
(l), (l), and (l) just as in Remark [H 

Note that even Generalized HCMS does not guaranties 
the existence of perfect compression as perfect compres- 
sion may simply does not exist. For example, there is no 
perfect compression for s = 3, n = 5, M = 3 as shown 
in Proposition [TJ 

VII. Universality of Generalized HCMS 

We are to prove that every perfect compression for 
Hamming sources S is equivalent to a generalized 
HCMS. We say two perfect compressions are equivalent 
(denoted by ~) to each other if and only if their null 
spaces can be converted to each other through the steps 
of the null space shifting as to be described in Lemma 
Q] Since each step of null space shifting is invertible, 
the term "equivalent" is mathematically justify. The set 
of perfect compression does form equivalent classes. 
The objective of this section is to show the following 
theorem. 

Theorem 4. Every perfect compression is equivalent to 
a generalized HCMS. 

To prove Theorem |U we will introduce and show 
several lemmas to achieve our goal. 

Lemma 5. Every 2-source perfect compression is equiv- 
alent to a Hamming code. 



and P is bijective over the set of all length-sn vectors 
with weight 1. 

After knowing (vi,-- - , v s ), we can compute 
Gi(b),--- ,G s {b) and Ci(b),--- ,C s (b). This in turn 
gives us T(b) and Y(h), respectively, (the latter is again 
by Remark [T|). So we have Rb. Since R is invertible, we 
can get back b and thus all sources. 

The second claim is apparent by simple counting. ■ 

Example 2 (Generalized HCMS of three sources of 
length-1). Let us revisit Remark [2] For the case s = 3, 
n = l, and M = 3. consider the Hamming matrix 

P = ( 011 ) = [Q1Q2Q3} we must get d = C 2 = 



(by (O) Proof: Let s = 2. If (Hi, H 2 ) is a perfect compres- 
sion, then we can let N\ = {0} and K = null(ffi) 
and form (H[,H' 2 ) under Lemma 1. Having {0} as 
(by ©null space, H[ can be any invertible n x n matrix and 
we can set H[ to the identity matrix without loss of 
generality. Meanwhile, H' 2 is a full rank m x n matrix 
with 2 m = n + 1. Since the columns of H 2 must 
be nonzero and different from each other (because i- 
th column = j-th column <^ H' 2 ei 



H' 2 ej, where 



e; = [0, -- ,0,1, 
press S because e 




(H[,H 2 ) fails to corn- 
inputted to encoder 2 are 
no longer distinguishable by the encoder's output), H' 2 
is unique up to a permutation of columns. Therefore, 
H' 2 is a parity check matrix of the Hamming (n,n — m) 
code. Conversely, we can construct (Hi, H 2 ) (up to their 
null spaces) from (H[,H 2 ) by Lemma [TJ That means 
any perfect compression is equivalent to Hamming code 
under Lemma Q] ■ 



s 



Lemma 6. Given a perfect compression 
(H X ,H' 2 , ■ ■ ■ ,H' S ), there exists (H X ,---,H S ) ~ 
(H[,--- ,H' S ) s.t. nullHi<T\ ...<T\ null Hi_iC\ null H i+ iC\ 
... n nullH s = for 1 < i < s. 

Before we proceed with the proof of Lemma |6l we 
will introduce a fact necessary for the proof as follows. 

Fact 1. For vector spaces U, V, and W, it is easy to 
show that 

(v + u) n w c (v n w) + u if u c w. (12) 



Proof: Let v G V, u G U that v + u G W. Then 
ueUcW^-veW^veVnW. As a result 

v + u g (v n w) + u. m 

Proof of Lemma @ Let Ri = f] nullf/j for 
1 < i < s. We have 

Ri C nullHj, for 1 < j < s and i 7^ j, (13) 

and 

R i nR k = f] nullHj = for i ^ k, (14) 

l<j<s 

where (a) holds because the perfect compression 
(H[, H 2 , ■ ■ ■ ,H' S ) must be injective and hence the in- 
tersection of all of their null spaces must be 0. 

By ( fT3l ) and (fT~4b . there exist a space N s that we can 
decompose 



nullH' s = N s Ri 



R.i 



(15) 



Again by (fT3l) and (IT4b together with Lemma [T] we 
can form an equivalent perfect compression by first 
moving the whole Ri from nullH' s to nullHj for i runs 
from 1 to s — 1 and get 

Ni = nullH' i ®R i ,l<i<s (16) 

Then if we let Hj be a surjective matrix with 

nullHj = Nj for 1 < j < s, (17) 

we have {H u --- ,H B )~{H[ r -- ,H' S ). 

Now let Li = C\i<j<s\j^i N i for 1 < i < s, we still 
need to show Lj = 0. 

By symmetry, all we need to show is L\ = N2 fl 
N 3 n • • • n iV s _i n iV s = 0. By (fT6l we have N 2 C 
nullHj © i?2- Suppose iV 2 n • • • n N k C (nullHj D 

• • • n nullH' k ) + (i?2 H h .Rfc) for a fe < s - 1. Then 

N2 (~l • • • n iVjfc+i is a subset of 



((nullH' 2 n • • • n nullH' k ) + (i? 2 H h -Rfc)) 

n(n«iZflfc +1 + i2fc+i) 



by induction hypothesis. 

By CCD) R2 H h .Rfc C nullH' k+1 C nullH' k+1 + 

Rk+i- So we can apply Fact Q] on (fT8T ) and thus obtain 
N 2 n - ■ - DN k+1 as a subset of {(nullH' 2 r\ ..DnullH k )n 
(nullH' k+1 + + (i? 2 + ■ ■ ■ + R k ))- Apply Fact 

[TJ once more with V = nullH' k+l ,U = R k +i,W = 
nullH' 2 n • • • fl nullH' k , we get (c.f. CTJ> for U C W) 
iV 2 n- • -niVfe+i is a subset of (nullH 2 n- ■ ■r\nullH' k+l )+ 

R2 + R3 H h -Rfc+i- By induction we get iV 2 n • • • n 

iV s _i C (nullH 2 n • • • n nullH' s _ x ) + i? 2 H h i? s -i- 

Lastly, iV 2 n • • • n N s 

C {(nullH 2 n • • • n nullH' s _ x ) + i? 2 H h i? s -i) 

fl nullH' s 

(b) 

C (nullH 2 n • • • n nullH' s ) + R 2 -\ h i? s -i 

(c) 

= i?l + i?2 + " " " + Rs—li 

where (a) is due to JV S G nullH' s (c.f. (031)). (b) is due 
to Fact [1] and (c) is from the definition of R x . 

Thus, N 2 n • • • n N s C H h iZs-i) n iV s = 

{0}, where the last equality is from the construction of 

Ng (C.f. (21). ■ 

Lemma 7. Given the coding matrices, (H x , • • • , i? s ), 0/ 
a perfect (s,n, M)-compression, we can form a perfect 
(2, sn, M + (s — l)n)-compression with coding matrices 
(X, J), where 



(19) 







/ 





• 


•■ 





°\ 


X = 







/ 


• 


•■ 
































• 


•■ 




1) 



is a {s — l)n x sn matrix, and 





(Hi 


• 




J = 





H 2 ■ 


■ 






■ 





(20) 



is a M X sn matrix, and L denotes the n x n identity 
matrix. 

Proof: Since 2 n (sn + 1) = 2 M implies 2 sn (sn + 
1) = 2 A/ +( s ~ 1 ) n , we only need to show how to retrieve 
the input vectors. Let us decompose any pair of the 

/b,\ 

input vectors for X and J. respectively, into b * 



% + Vl' 



(18) 



Vv 

and I ■ ■ ■ I , where bj's are n-entry vectors, Vj's 

are also n-entry vectors but restricted to the condition 
|vi| + • • • + |v s | < 1, where | • [ maps an element in 
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Z 2 to its norm in Z by counting the number of nonzero 
components. 

From the output of X, we will get t>i + b 2 ,b 2 + 
b 3 ,-- ,b s _i + b s . Thus we can obtain bi + 
b 2 , bi + b 3 , • • • , bi + b s and H 2 (b 1 + b 2 ), F 3 (bi + 
b 3 ),--- ,H s (bx + b s ). 

From the output of J, we will obtain -£f 2 (b 2 + 
v 2 ), F 3 (b 3 + v 3 ), ■■■ ,H s (b s + v s ) and #i(bi + vi). 
Combining the results, we get -ffi(bi + vi),// 2 (bi + 
v 2 ),--- ,# s (bi + v s ). Since (Hi,-- - ,H S ) is a per- 
fect compression, we can compute bi,vi,v 2 ,--- ,v s . 
Together with the output of X, we can retrieve all 
bi,b 2 , • • • ,b s and vi, v 2 , • • • , v s . ■ 

Before we finally proceed to the proof of Theorem [4] 
we need to present one more fact. 

Fact 2. For vector spaces V, U, and W, (V@U)C\W = 
(VnW)®UifUcW. 

Proof: From (O, we know that (V + U) D W C 
(V n W) + U. Now let v G V n W, u G U C W. Then 
v+u G W and v+u G V+U. Therefore (VC\W) + U C 
(V+U)DW. Lastly, we notice that V(~)U = (VnW)nU 
as U C W. Hence FnC/ = {0} iff (YfW)n£7 = {0}, 
that justifies the direct sum signs. ■ 

Proof of Theorem @ - 
By Lemma [6l we can restrict our attention only to 
perfect compression whose coding matrices (Hi, ..,H S ) 
satisfy 

nullHiPinullH2 ■ ■ -OnullHi-ir\nullHi+iC\- ■ -OnullH s 

(21) 

for i 7^ s without loss of generality. 

We can also generate X and J according to (fT9l ) and 
(l20b . Then Lemma [7] shows that (X, J) is a perfect 
(2, sro, M + (s — l)n) -compression. Therefore nullX n 
nullJ = 0. Then Lemma [T] tells us that two surjective 
matrices with null spaces {0} and nullX © nullJ, 
respectively, are also a perfect compression. By the proof 
of Lemma [51 the first matrix is an invertible matrix and 
the second one is a (M + (s — l)n — sn =)M — n-bit 
Hamming matrix P. 

We have 

ratZZP = nullX © nuMJ (22) 

and 

nullX = {(c,--- ,c)*|cGZ 2 "}; 
nullJ = {(ni, • • • , n s )*|rij G nullHi}. 
Partition P into 



because nullX C null P. 

Secondly, nullJ C nullP implies 

nullHj C nullQj (26) 

for 1 < j < s. Moreover, let 

6j G nullQj, (27) 

we have (0, • • • , 0,bj,0, ■■■ ,0)* G nullP and we 
can decompose it into (c, • • • , c) + (c, • • • , c, c + 
bj, c, • • • , c) by d22l with 

(c, • • • , c, c + bj, c, • • • , c) G null J (28) 

So c G nullHi n nullH 2 Pi • • • D nullHj-i n nullHj + i n 
• • -r\nullH s and c+bj G nullHj. By (l2Tb . we have c = 
if j 7^ s and bj G nullHj. Hence nullQj C nullHj. 
Together with (1261 ). we get 

nullHj = nullQj for j / s. (29) 

Recall that Y is a row basis matrix of 

" Qi " 

^ 2 (30) 

_Qs~i_ 

and hence 

nullY = nullQif\...f]nullQ s -i = nullHif\...C\nullH s ^i 

(31) 

bjy d29l . Now we will show that nullQ s = nullH s © 
nullY. 

nullQ s C nullH s © nullY: Equations (1271) and d28l) 
are true for all j. In particular, when j = s and let 
b s G nullQ s , we have b s = c + b s + c, where 
c G nullHi n • • • fl nullH s -i = nullY and c + b s G 
nullH s by (l28l) . Therefore, b s G nullH s + nullY. Since 
nullH s fl nullY = nullHi n ... fl nullH s = 0, we have 
nullQs C nullH s © nullY. 

nullH s © nullY C nullQ s : Given b s G nullH s 
and c G ratZZY, (0, 0, • • • , 0, b s + c)* = (c, • ■ • , c)* + 
(c, • • • ,c,b s )* G nnZ/P (c.f. d22"ll . (1231 ) because 
(c, • • • , c) G nullX and (c, ■ ■ • , c, b s ) in nullJ. Thus 
= P(0, • • • , 0, 6 S + c)* = Q s (b s + c). Hence b s + c G 
nullQs and thus we have 

nullQs = nullHg © nullY (32) 



(23) 

Let ^4 be a subspace of Z2 such that 

nullHs @nullY @A = Z%. (33) 
Let T be a surjective matrix with 

nitZZT = nullHs © A (34) 



P=[QiQ2--- -Qs], (24) 
such that is a (M — n) x n matrix. We have 

Qi + Q 2 + ■ ■ ■ + Qs = (25) 
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We have 



nullT nullY = 



(35) 



Being a row basis matrix matrix (c.f. (1301)). Y is also 
surjective. We have 



R 



(36) 



is annxn invertible matrix. 

nullQ s n nullT = (nullH s ® nullY) n nu/ZT. Since 
nullH s C nullT (c.f. d34l)), we can apply Fact |2] to 
obtain 

nullQs n rraZZT = (nullT n raiMY) nullH s (37) 
= © nu»# s (c.f. 65)) 
= nullH s 

Denote Cj as a row basis matrix of Qj for all j, then 
we have 

nullCj = nullQj (38) 

for all j. 

Since Q s = Q x + Q 2 + ••• + (c.f. (E3)) and C s 
is a row basis matrix of Q s (c.f. (1381)), we have 



rowC s = rowQ s C rowY 



(39) 



(c.f. (ED)). Then 



T 

C, 



is a surjective matrix because its 



row vectors are linear independent, thanks to (1361 ) and 
&. 

( T 
Therefore, C\ , C 2 , • • • , C s _i, 



is a perfect 



compression as its encoding matrices are all surjec- 
tive and have the same null spaces of the perfect 
compression (Hi,-- - ,H s )'s (c.f. Lemma [Tj- More- 

T~ 

C s 



over Ci,C 2 , • • • ,C S 



which is equivalent to (Hi,- 



is a generalized HCMS 

HA 



Appendix 

Proof of Proposition \J} 

Denote S as the set containing all 3-terminal Ham- 
ming source of length 5. Let Hi,H2,H% be the three 
parity check matrices. Our strategy is to limit the null 
spaces of them by the fact that all elements in S need to 
have distinct syndromes. The limitation will eventually 
kill the possibility of the existence of Hi, H2, H%. 

Denote the null set of a matrix A as null^) = 
{u|Au = 0}, where is an all zero vector. Further, 
denote ej as the length- 5 binary column vector that has 
i th component equal to 1 and the rest of its components 
equal to zero. 



We may assume the number of row in H\ is smaller 
than or equal to the other's. In other words, H\ has 
at most 3 rows. Hence null(ffi) has at least two de- 
grees of freedom. Regardless the values of H2 and 
H3, null(i/i) cannot contain any e^. Otherwise, both 
(ej,0, 0) and (0,0,0) that are in S will get the same 
outputs (yi,y 2 ,y3) = (0,0,0). Similarly, null(ffi) 
cannot contain ej + ej neither, otherwise both (ej, 0, 0) 
and (ej,0,0) (in 5) will get the same outputs because 
H\ei = Hiej. Thus null(//i) can only be span(ej + 
ej + efc,e,, + e m + e n ), where the letters i,j,k,m,n 
are different to each others, i.e. null(ffi) = {0,ej + 
ej + e fe , e.j + e m + e n , ej + e k + e m + e n } for some 
i, j, k,m,n. Other structures such as higher dimension 
will contain forbidden elements. As the dimension of the 
null spaces of 1 x 5 and 2x5 matrices are all greater 
than 2, Hi has to have at least 3 rows. Thus both H2 
and H3 also have three rows. It excludes the possibility 
of perfect asymmetric SW codes (at rate [2/5, 3/5, 4/5], 
for example). So we can focus only on the symmetric 
case from now on. 

From the above discussion, null(i^i) has to contain 
0, two "3e" vectors and one "4e" vector, and no more. 
Similarly, null(i^2) and null(f//3) get the same structure. 

Without lose of generality, we can write null(i^i) = 
{0,ei+e2 + e3,ei + e 4 +e5,e2 + e 3 + e4+e5}. Suppose 
null(i^2) contain ei + e2 + e3, then of course null(if3) 
cannot contain ei + e2 + 63. Otherwise, (0, 0,0) G S 
and (ei+e2 + e 3 ,ei+e2 + e 3 ,ei+e2 + e 3 ) G S get the 
same output (0, 0, 0). But null(i/3) cannot contain e» + 
ej + efc, i,j G {1,2,3}; k G {4,5} (i / j). Otherwise 
(ei+e2+e 3 ,ei+e2+e 3 ,ej+ej) G S and (0,0, e fc ) G S 
share the same output as well. So the "3e" vectors of 
null(ff3) can only be two of ei + e 4 + es, e2 + e 4 + e5, 
and e3 + e 4 + es. Unfortunately, any pair of them sum 
up to a "2e" vector instead of a "4e" vector. Therefore, 
there cannot be a common "3e" vector shared between 
any pair of the null spaces of Hi,H2, and H3. 

Now, suppose null (^2) contains the same "4e" vector 
e2+e3+e 4 +e5 as null(//i) does. Then nul^H^) cannot 
contain any "4e" vector. Let the "4e" vector of nuH(H^) 
be ei + e 2 + e 3 + e 4 + e 5 - e^, j G {2, 3, 4, 5}. Then 
(e 2 + e 3 + e 4 + e 5 , e 2 + e 3 + e 4 + e 5 , [1, 1, 1, 1, 1] T ) and 
(0, 0, ej) shares the same syndrome. Thus, any pair of 
null spaces of Hi,Hi, and H3 cannot share a common 
"4e" vector as well. 

Hence, without loss of generality, we can write 
nuil(i7 2 ) = {0,e2 + ei + e 4 ,e 2 + e3 + e 5 ,ei + e3 + e4 + 
65}. Then, there are only three different possibilities for 
the "4e" vector of null (#3): 

1) ei + e 2 + e 4 + e 5 ; 

2) ei + e 2 + e 3 + e 5 ; 
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3) ei + e 2 + e 3 + e 4 . 

Case 1 does not work because (ei + e 4 + es, ei +e3 + 
e 4 + e5, ei + e 4 + e§) G S and (0, 0, e 2 ) G S shares the 
same syndrome. 

Case 2 does not work neither because (ei + e 2 + 
e 3 ,ei + e 2 + e 3 + e 5 ,ei + e 2 + e 3 + e 5 ) G S and 
(0, ei, 0) G S share the same syndrome. 

Finally, case 3 fails as well since (ei + e 2 + e 3 , ei + 
e 2 + e 3 + e 4 , ei + e 2 + e 3 + e 4 ) G 5 and (0, e 3 , 0) G 5 
share the same syndrome. ■ 
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